Comment on ``Quantitative performance metrics''

Introduction Conclusions References Tables Figures Back Close Full Screen / Esc Atmospheric Chemistry and Physics Discussions This discussion paper is/has been under review for the journal Atmospheric Chemistry and Physics (ACP). Please refer to the corresponding final paper in ACP if available. Abstract Introduction Conclusions References Tables Figures Back Close Full Screen / Esc Abstract This comment focuses on the statistical limitations of a model grading, as applied by D. Waugh and V. Eyring (2008) (WE08). The grade g is calculated for a specific diagnostic , which basically relates the difference of model and observational data to the standard deviation in the observational dataset. Monte Carlo simulations show that 5 this method is not leading to statistical significant gradings. Moreover, the difference between two models is hardly significant. The results of the statistical tests performed in WE08 agree with our findings. However, most of those tests are based on special cases, which implicitely assume that observations are available without any errors and that the interannual variability of the observational data and the model data are equal. 10 Without these assumptions the grading becomes basically insignificant. We further show that the inclusion of confidence intervals into the grading approach is necessary, since it has the potential to change the grading results drastically.